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Abstract 

o : In the present paper we study properties of roots of characteristic polynomials for the linear 

recurrent formulae (LRF) that govern time series. We also investigate how the values of these 
roots affect Singular Spectrum Analysis implications, in what concerns separation of components, 
' SSA forecasting and related signal parameter estimation methods. The roots of the characteristic 

polynomial for an LRF comprise the signal roots, which determine the structure of the time series, 
and extraneous roots. We show how the separability of two time series can be characterized in 
, terms of their signal roots. All possible cases of exact separability are enumerated. We also examine 

' properties of extraneous roots of the LRF used in SSA forecasting algorithms, which is equivalent 

to the Min-Norm vector in subspace-based estimation methods. We apply recent theoretical results 
for orthogonal polynomials on the unit circle, which enable us to precisely describe the asymptotic 
" ^ , distribution of extraneous roots relative to the position of the signal roots. 

ro . Keywords: Singular Spectrum Analysis; SSA; separability; linear recurrent formula; LRF; continu- 

^ : .,0. ..p.e ..... „„ .e 

O ; 1 Introduction 
O 

In the theory of Singular Spectrum Analysis [11, 17] the time series that are governed by a linear 
recurrent formula (LRF) are of great importance. For these time series F/v = (/o, • • • , In-i) there 
^ ' exist coefficients such that the relation 

_ p-l 

fp+n = ^ akfk+n, (1) 

ao 7^ 0, holds for all appropriate n; these time series are called the time series of finite difference 
dimension. In the present paper we consider complex-valued time series and LRFs with complex 
coefficients. 

The roots of the characteristic polynomial A{z)'^=z^ — ap-iz^~^ + . . . + aiz + ao of the LRF (1) 
can be divided into two groups: the signal roots which determine the representation of the time series 
as a sum of polynomially modulated exponential signals 

m 

fn = Y.Pk{n)\l, (2) 

k=l 

and other roots, called extraneous roots. In this paper we show how the signal roots of the LRFs deter- 
mine several properties of time series, in what concerns the Singular Spectrum Analysis. Specifically, 
we consider separability and the behaviour of extraneous roots for the LRF used in SSA forecasting. 
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The latter LRF is of particular interest since the extraneous roots affect the results of SSA continua- 
tion. The obtained results also concern subspace-based methods of signal processing [4, 9, 12, 19] (in 
particular, the Min-Norm method), which share their structure with SSA. 

The first part of this paper is devoted to a novel outlook at separability [17, Ch. 6]. A new criterion 
of exact weak separability for the time series of finite difference dimension is introduced. This criterion 
enables us to present all the standard examples in a unified manner, elucidating the meaning of the 
conditions of separability in these examples. Moreover, it enables us to enumerate all possible cases 
of exact weak separability for arbitrary time series; this is also a new result. In fact, we develop a 
criterion for the one-sided separability, which has been recently shown to be important for the SSA 
theory [23]. The criterion is simple, illustrative and based solely on the signal roots of the time series. 
Specific features of the case where the time series is real are clarified as well. 

The second part of the paper deals with the LRF used in SSA forecasting (shortly, the SSA LRF), 
see [17, Ch. 5]. This LRF coincides with the Min-Norm prediction vector [4, 9, 12]. Properties 
of the extraneous roots of the SSA LRF are examined in the noise-free case. Recently, the basic 
properties were proved by various authors. In particular, it was proved that all extraneous roots lie 
inside the unit circle and that the extraneous roots of the forward and backward SSA LRF coincide, 
see [5, 15]. The asymptotic distribution of the extraneous roots was independently addressed in [7, 16], 
but only some particular cases were considered. In the present paper we show the correspondence 
between the extraneous roots of the SSA LRF and orthogonal polynomials on the unit circle [IS]. This 
correspondence enables us to provide elegant and short proofs for the basic properties of the extraneous 
roots and to describe the asymptotic distribution of the roots. By the way, this approach was used in 
[7], but in the present paper we try to provide a more accurate, comprehensive and integral exposition. 
We describe the asymptotic distribution for the general case of time series of finite difference dimension 
with the help of the most recent advances in the theory of orthogonal polynomials [20, 21]. 

Both parts of the present paper are based on the same ground of a revised theory of time series of 
finite difference dimension. In fact, this revision can be useful in the SSA theory on its own. Surpris- 
ingly, the two parts of the present paper are linked from another side: the approximate separability 
of certain time series can be somehow described with the help of the distribution of extraneous roots 
of a specific SSA LRF. 

This paper is organized as follows. In Section 2 we provide all necessary background on the linear 
recurrent formulae and continuation, the time series of finite difference dimension and their trajectory 
spaces. Section 3 is devoted to exact weak separability. In Section 4 we investigate the SSA forecasting 
LRF and the properties of its extraneous roots. At the end of Section 4 we discuss the connection 
between the asymptotic distribution of extraneous roots and separability; we also make practical 
conclusions and examine the behaviour of roots in the presence of noise as well. 

2 Basic facts and notation 

2.1 Time series of finite difference dimension and signal roots 

Let No denote the set of all nonnegative integers. An infinite time series 

Foo = (/o,/i,...), fneC, (3) 

is said to satisfy a linear recurrent formula (LRF) of order p if there exist coefficients a^, . . . , ap_i € C 
such that the relation 

p-i 

fp+n = ^ a-kfk+n (4) 
fc=0 

holds for all n G Nq. Note that in the case p = we have /n = for all n € Nq. Once a time series 
satisfies an LRF (4), its form can be described by the roots of the characteristic polynomial of the 
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LRF 

A{z) = — ap-iz^~^ — ... — aip — ao- (5) 

Theorem 2.1 ([11, Th. 3.1.1]). Assume that an infinite time series Foo satisfies an LRF (4) with 
ao 7^ 0. Then it can be represented as 

m 

fn = Y.Pk{n)\l, (6) 

k=l 

where \k S C\{0} are distinct numbers, and P/^ are non-zero polynomials. All in the representation 
(6) are roots of the characteristic polynomial A{z), with multiplicity not less than Uk'^ degPk + 1, where 
deg • is the degree of a polynomial. 

The coefficients of Pk are determined by the first d values of the time series, where d is defined as 

d = ui + . . . + Vm < p. (7) 

Remark 2.1. If a time series admits a representation of type (6), then this representation is unique. 
This fohows from the hnear independence of the time series of type Qn = n'^A" for different A G C \ {0} 
and k G Nq. 

For a time series of type (6), by Remark 2.1, one can unambiguously define the polynomial 

p{zf^{z-\ir^-...-{z-\mr"^ = .g. 

= PdZ'^ + ...+PIZ+PQ, 

where P(i = ^. This polynomial is called the characteristic polynomial of the time series. The charac- 
teristic polynomial determines the set of all LRF that are satisfied by the time series. 

Theorem 2.2. Let Fqq be a time series of the form (6). Then any polynomial 

B{z) = brz'- + ...biz + bo (9) 

of degree r (i.e. br ^ 0) is a multiple of the characteristic polynomial (8), i.e. B{z) = P{z)Q{z), if 
and only if the time series satisfies the LRF 

r-1 , 

/„+, = ^--^/„+fc, nGNo. (10) 



fc=0 



r 



r-1 

Proof. I I If F satisfies an LRF fr+n = bkfk+n, which has the characteristic polynomial B{z) = 

(r+O-l 

z^ — br-iz^~^ — ... — bo, then it satisfies the LRF = ^ bkfk+n = 0, which corresponds to 

k=l 

B{z)zK Therefore, for any A{z) = B{z)Q{z), F satisfies the corresponding LRF. 

The proof that a time series of the form (6) satisfies the LRF with the characteristic polynomial 
P{z) can be found in the proof of [14, Th. 3.1.1]. 

I <^ I Note that if i^oo satisfies the LRF with the characteristic polynomial Q{z) = S{z)z'"^ , 5'(0) ^ 0, 
then the time series Goo = {fm, fm+i, ■ ■ ■) satisfies the LRF corresponding to S{z). Let R{z) = 
GC'D{P{z), S{z)) be the greatest common divisor of P{z) and S{z). Then R{z) can be represented 
in the form R{z) = P{z)c{z) + S{z)d{z) (see [10, Ch.III, §17]), and by |^ part. Goo satisfies the LRF 
with characteristic polynomial R{z). If R{z) is not P{z) then R{z) = (z — Ai)'^^ • . . . • (z — Xm)'^"^ with 

m 

dk ^ ^k for all k and at least one di < ui. Then by Theorem 2.1 g^ = ^ <5a:(?T')A^, where Qi{n) has 

k=l 

the degree less than dk — 1. By the linear independence of time series n^X^ we obtain the contradiction 
with the representation (6). □ 
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Remark 2.2. Theorems 2.1 and 2.2 establish the one-to-one correspondence between the time series 
of type (6) and the time series satisfying at least one LRF (4) with non-zero last coefficient (ao 0). 

Now assume that a time series F^o satisfies an LRF (4) with ao 7^ 0. By Theorem 2.1 it has the 
representation (6) and the characteristic polynomial (8) is uniquely determined. By Theorem 2.2, the 
relation 

A{z) = P{z)V{z) (11) 

holds. Moreover, the time series satisfies all LRFs with characteristic polynomials of form B{z) = 
P{z)Q{z), and hence the polynomial V{z) (11) (and its roots) has no effect on the form of the time 
series. Thus, the p roots of the characteristic polynomial A{z) can be divided into two groups: 

1. the d signal roots (i.e. the roots of P{z)), which determine the structure of the time series, 

2. the p — d extraneous roots, 

where d is defined in (7). We also say that the signal roots of A{z) are the signal roots of the time 
series and Vk sue their multiplicities. 

The number d of signal roots has an important interpretation in terms of LRFs; this interpretation 
follows from Theorem 2.2. 

Corollary 2.1. If a time series F^q satisfies an LRF (4) with oq ^ 0, then the LRF corresponding to 
the characteristic polynomial (8) of F^o 

d-l 

fd+n = — ^Pfc/fc+n (12) 
k=0 

has the minimal order d among all LRFs satisfied by F^o ■ 

Note that Corollary 2.1 is a characterization of P{z), and can be taken as an alternative definition 
of the characteristic polynomial P{z). It also validates the following notation. 

Definition 2.1. We say that Fao is a time series of finite difference dimension (an f.d.d. time series) 
if it satisfies at least one LRF (4) with oq ^ 0. The degree d of the characteristic polynomial, defined 
in (7), is called the difference dimension of Foo- 

Remark 2.3. The case of LRFs with oq = can be considered within the same framework along 
with generalizations of Theorems 2.1 and 2.2, but we omit this consideration. 

For clarity, let us consider a real-valued time series F^o of finite difference dimension. By Theo- 
rem 2.1, it has the form 

s' 

f^ = Y^Pi{n)pf+ (13) 
1=1 

s 

+ XI pf cos{2TTUin + ifi), 

l=s'+l 

where ui, pi are distinct, \uji\ < 0.5 and Pi are real polynomials of degree ni — 1. If we denote A; = pi, 
vi = ri for I < s' and X2i-s'-i = /Oje^™', X21-S = fte"^™', i/2Z-s'-i = ^21-3 = ni for s' < I < s, then 
we obtain the representation (6) with m = 2s — s' modulated exponents. 
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2.2 Hankel matrices and trajectory spaces 

Let 

F = FN = {fo,...jN~iy e 



(14) 



be a (finite) time series. The Hankel matrix generated by the time series (or the trajectory matrix) is 
the matrix 



X(^)(F;v) 



dcf 



/ /o 


/l 


/2 


• • • fK-1 ) 


/l 


/2 


/3 




/2 


/3 


/4 


. . . Ik+i 



(15) 



where the parameter L is cahed the window length, 1 < L < N and K = N — L + 1. 

Definition 2.2. If F/v is a subseries of an infinite time series (3) of difference dimension d < N/2, 
then Ftv is called a time series of (finite) difference dimension d (with characteristic polynomial P{z)). 

This definition agrees with the definition given in [17, Ch. 2], see remarks at the beginning of 
Section 2.3. In particular, the following theorem states that the time series of finite difference dimension 
are time series of finite rank, see also [17, Ch. 5, Prop. 5.4]. 

Proposition 2.1. Let Fn be of difference dimension d. 

1. For the window length L such that d<L<N — d+1 the trajectory matrix (14) is of rank d. 

2. IfL<dorL>N — d+1 then X^^^ has maximal possible rank (L or N — L + 1, respectively) . 

The proposition immediately follows from [13, Ch. XVI, §10, Th. 7] and its corollary. 
Let us show how the structure of a time series is connected to LRFs which are satisfied by the 
time series. The structure of a time series in SSA is described by its trajectory space 



£(^)(F^)1|fspan(x|^\ . . . ,X]^^) C C^ 



where 



X. 



(/,_!,..., /i+L_2)^ l<i<K, 



(16) 



are the columns of the matrix X^^^ = 'K^^\Fj\f), see [17, Ch. 1] for a detailed discussion. In what 
follows, the following subspace of is very useful. 

Definition 2.3. The relations space is defined as 



where £ denotes the complex conjugation of £ and £^f^ is the orthogonal complement to the trajectory 
space. 

The relations space consists of all linear relations on rows of X*-^\ Indeed, the vector 

{ao,...,ar,0,...,Oy, Ory^O, 

belongs to ^K^^^ if and only if 

r-l 



f 



n+r 



■Y^Jlf^^j,, 0<n<iV-L + l. 



(17) 



fc=0 
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Remark 2.4. In complex vector spaces the inner product involves complex conjugation. This explains 
the presence of the conjugation in the definition of the relations space. If F/v is real-valued, then 



The following proposition shows that the relations space of a time series of finite difference dimen- 
sion is generated by all LRF of order less than L, satisfied by its infinite time series (c.f. Definition 2.2). 



Proposition 2.2. Let Fj\[ be a time series of difference dimension d with characteristic polynomial 
P{z) (8). For the window length L, d<L<N — d+1, we have the following. 



1. The columns of the Lx(L — d) matrix 



p(i) 



f PO 

■ Po 

Pd ■■ 

Pd 

\0 ... 



\ 


Po 

Pd ) 



(18) 



form a basis of the space 



2. A vector B = {bo, . . . , bi-iY belongs to if and only if B{z) 
a multiple of P{z). 



bi-iz^^^ + ... + biz + bo is 



Proof. Since pd ^ 0, the columns of the matrix P (18) are linearly independent. By Theorem 2.2, the 
time series Fjsf satisfies the LRF (12) and the L — d columns of P belong to By Proposition 2.1, 

the dimension of ^^^^ is L — d, and hence the columns of P form a basis of this space. □ 

Remark 2.5. By Remark 2.4, for a real time series the basis given in Proposition 2.2 is a basis of 
£,^^^ as well. 

We also note that a similar basis can be introduced for arbitrary (not only f.d.d.) time series 
(see [()]) and can be very useful in the SSA theory. In the present paper we demonstrate several 
applications of this basis in the case of time series of finite difference dimension. We will also need a 
("Vandermonde"-like) basis of the trajectory space. 

Proposition 2.3. Let F^' be an f.d.d. time series with the characteristic polynomial defined in (8). 
Let also d<L<N — d+1. Then a basis of £'^^) is given by the vectors 



^0 (Ai), . . .,i2^\xi), . . . , £i(A^), . . . , rj--\x^), 



where 



£^(A) = |^(l,A,A^...,A^-i) 



L-IM 



(0,...,0,A;!,, 



' (L-fc-l)!' 



iL-fc-l\T 



Proof. Let (Goo) be the formal continuation (as an infinite time series) of ^^(A), e.g. 



A,0 



\\i ,fc 



(1,A,A2,...). 

< /c < z/j, satisfies the LRF (12) and all LRFs with 



(Goo) 

Then by Theorem 2.2 any time series (Gc 
characteristic polynomials of type P{z)Q{z), where Q{z) ^ 0. Therefore, by the second assertion of 
Proposition 2.2, each vector ^f,(Aj) is orthogonal to Since these d vectors are linearly independent, 
the assertion is proved. □ 
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2.3 LRFs and continuation 

In this section we discuss the time series that can be continued within the SSA framework. A time 
series defined in (14) admits the (forward) L- continuation (is L-continuahle) if there exists unique 
a € C such that 2^^\Fn) = £(^)(/o, . . . ,/jv-i,a), see [17, Ch. 5] for details on continuation. First, 
we show a connection between these time series and time series of finite difference dimension. 

Proposition 2.4 ([IT, Th. 5.4]). If a time series F/v satisfies some LRF 

fn+do = CLkfn+k, < n < N - do - I, (19) 
fc=o 

with do ^ min(L — 1,K), then it is L-continuahle and the continuation is achieved by the same LRF 
(i.e. (19) holds for n = N — do if we set fN = a). 

Remark 2.6. By Proposition 2.4 one can continue F/v to an infinite time series F^o = {fo, fi, ■ ■ ■), 
which satisfies (19) for ah n (i.e. Fqo is of finite difference dimension). This fact is the base of SSA 
forecasting, see [17, Ch. 2]. 

The condition (19) means that the time series F/v is f.d.d. in the sense of [17, Ch. 2], see also 
a remark after Definition 2.2. However, Remark 2.6 together with the following assertion shows that 
(19) is equivalent to the definition adopted in the present paper. 

Corollary 2.2. Any time series Fn of difference dimension d is L-continuahle ifd<L<N — d+1. 

Evidently, the infinite continuation (see Remark 2.6) of a finite subseries F/v of an f.d.d. time series 
Foo (with d < N/2) coincides with the original time series Foo- This observation removes the ambiguity 
from Definition 2.2: a finite time series -F/v of finite difference dimension cannot be a subseries of more 
than one infinite f.d.d. time series due to the uniqueness of continuation. 

The following result, which is the converse to Proposition 2.4, can be found in [(i, Ch. 5]. 

Proposition 2.5. // a time series F/v is L-continuahle, then there exists do < min(L — 1,K) such 
that F/v satisfies an LRF (19). 

Proof. The proposition is a direct consequence of [6, Th. 5.6] and [G, Prop. 5.8]. □ 

For convenience, we recall the well-known necessary and sufficient conditions for F/v to be L- 
continuable. 

Proposition 2.6 ([17, §5.3]). If Fn is L-continuahle, then cl ^ 2.^^\Fn), where eL^^{Q, . . . , l)'^ G C^. 

Proposition 2.7 ([17, Th. 5.4]). Let L < N/2. If cl ^ q'-^^Fn), then Fn is L-continuahle. 

We also need the notion of the backward L-continuation. 

Definition 2.4. A time series F/v admits the backward L-continuation (it is backward L-continuahle) 
if there exists unique a S C such that £^^\Fn) = £^^\a, fo, ■ . . , fN~i)- 

It is clear that the backward F-continuation is equivalent to the forward L-continuation of the 
reversed time series F/v =^(/Af-i5 ■ ■ ■ , fo), and the following assertion holds. 

Corollary 2.3. 1. If Fn is backward L-continuahle, then ei ^ £(^)(F/v), where ei =^(1, . . . ,0)^ G 
2. Let L < N/2. If ei ^ £'^^)(F/v) then F/v is backward L-continuahle. 
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We conclude this subsection with a result which is complementary to Corollary 2.2 and will be 
very useful in the next section. 

Proposition 2.8. If Fn is both forward and backward L-continuable, then it is a time series of finite 
difference dimension d < min(L — 1, K). 

Proof. If Fat is forward L-continuable then by Proposition 2.4 there exist coefficients oq, • • • , Odi-i G C 
such that F/v satisfies 

fn+di = ^ a-kfn+k, 0<n<iV-(ii-l, 

k=Q 

where di < mm(L — l,K) < N/2. If ao 7^ then Fn is an f.d.d. time series by the definition. If ao = 
then we use the fact that i<jv is backward L-continuable and there exist coefficients bi, . . . ,bdi € C 
such that 

/n = X] ^kfn+k, 0<n<N-d2-l. 
k=l 

Without loss of generality, assume that d2 < di- Then there exists a constant a 7^ such that the 
last component of 

di—d2 

+ (0, -ai, . . . , -ad^_i, 1)'^ 

is non-zero. Then LV satisfies 

di-l 

fn+di = - X] — 0<n<N-di-l. 
k=o 

Since cq = a 7^ 0, the result follows from Theorem 2.1. □ 

3 Separability 

The main result of this section is a necessary and sufficient condition of separability in terms of charac- 
teristic polynomials and signal roots of f.d.d. time series. In particular, it enables us to enumerate all 
possible cases of weak separability and present standard examples (see [17, §6.1]) in a unified manner. 
Moreover, we develop the new theory for the one-sided separability, the results on the conventional 
separability are its consequence. 

3.1 One-sided separability criterion 

Definition 3.1. Time series F^\ F^'^ are called weakly left- separable (right-separable) if £.^^\F^^) _L 
£(^)(Fj^^^) (£(^)(Fj5,^^) ± respectively). 

For any vector B = {bo, . . . ,6l_i)^ G we denote by B{z) = bi^iz^'^ -|- . . . -|- biz -\- bo its 
generating polynomial. We shall also need a notation for the polynomial with conjugate coefficients, 
B{z) = bi-iz^^^ + ... + hz + b^. 
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Proposition 3.1. Assume that F^^ is a non-zero time series and F^"^ is a time series of difference 
dimension d with characteristic polynomial P^'^\z). Let L satisfy d < L < N — d+ 1 and {Ui, . . . , Ur} 
be an arbitrary basis o/£(^)(FjJ)). ThenF^^^ and F^ are separable if and only if P^'^\z) is a common 
divisor of Ui{z), 1 < i < d (in other words, all the signal roots of F^^ are common roots of all Ui{z), 
at least with respective multiplicities). 

Proof Note that _L if and only ii£(^\F^^'>) C £'-^\Fj^'^). The subspace inclusion 

can be reformulated as Ui € 5H'^^^(F^^) for all 1 < i < r (c.f. Definition 2.3). By Proposition 2.2, 
TTi G if and only if = P^^\z)Qi{z), where P^^\z) is the characteristic polynomial of 

(2) 

F^ and Qi{z) is a nonzero polynomial. This proves the proposition. □ 



Remark 3.1. Proposition 3.1 is valid even for 1<L<A^ — d+l, and thus it is valid for the essential 
case L < N/2. Indeed, if L < d, then by Proposition 2.1 dim(£^^(F^^)) = L — L = and only zero 

time series can be left separable from ijy • same time, any Ui{z) is of degree less than d and 

hence it cannot be a multiple of P^'^\z). 

In what follows we assume that L < N/2, unless indicated. Proposition 3.1 together with Re- 
mark 3.1 points out a method of finding all f.d.d. time series F^^ that are left-separable from given 

■ 

Proposition 3.2. Assume that F^^ is a non-zero time series and {Ui, . . . , Ur} is a basis of its trajec- 
tory space £,^^\F^^), L < N/2. Let fii, . . . ,fii be all distinct nonzero common roots ofUi{z),..., Ur{z) 
and di, . . . ,di be their common multiplicities, i.e. d^ is the minimal multiplicity of fi^ in polynomials 
Ui{z). Then all f.d.d. time series that are left-separable from F^^ are given by 

I 

/P=E^'^(")/^^' (20) 

k=l 

where Qki^) are (possibly zero) polynomials of degree less than dk and not all Qk are zero. 

Proof. Polynomials Ui{z) may have an additional zero root /ip = of multiplicity do G Nq. Then the 
polynomial 

R{z) = {z-f,if^-K..{z-fiif'-'z^'^ 

is the greatest common divisor of Ui{z). Therefore, by Remark 3.1, and F^'^ are left-separable if 
and only if the characteristic polynomial P^'^\z) is a divisor of R{z). By Theorem 2.1, the time series 
having this property are precisely the time series of form (20). □ 

Remark 3.2. The statement of Proposition 3.2 does not depend on the choice of the basis. Indeed, 
if ;U is a common root for the set of polynomials {Ui[z), . . . , Ur{z)}, then it is a common root of all 
linear combinations of Ui{z). 

Let Fj^^ be an f.d.d. time series of the form (6). Then a basis of £(^)(F^^^) is given by the vectors 
P'j^{\k) from Proposition 2.3, l<A;<m, 0<i< f^, and we can use Proposition 3.2 to determine all 
time series that are left-separable from f'^\ Moreover, we can check the left separability condition 
separately for each summand in the representation (6). 

Remark 3.3. Let F^'' be a time series of type (6). Then an f.d.d. time series F^^ is left-separable 
from F^'' if and only if it is left-separable from each Pfc(n)A^ given in (6). 

Indeed, each summand Pk{n)X^ has the vectors £^(Afc), < i < z^^, as a basis of its trajectory 
space, and the conditions for the signal roots of to be separable from F^'^ coincide with the set 
of conditions for the separability from each summand of F^'^ . 
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3.2 Separability of f.d.d. time series 

By Remark 3.3, to study the separability of arbitrary f.d.d. time series it is sufficient to consider tlie 
separability from a time series Pi(n)A", i.e. the time series with only one (possibly multiple) signal 
root. We are going to examine this case gradually in separate examples. In the present subsection we 
again assume L < N/2. 

Example 3.1. [Separability from a constant] Let F^^ = c 7^ 0. Then the space £,^^\F^^) is spanned 
by the single vector (1, . . . , 1)''^ G C^. Consider its generating polynomial 

V^i(z) = z^-i + ... + l = ^^. (21) 

The roots /.ifc of VFi(2;) are simple; they are the Lth roots of unity, excluding 1: 

/2TTik\ 

/ifc = exp ( — ^ I , < A; < L. 
In Fig. 1 the roots fik are represented by dots. 




Figure 1: Roots of the series left-separable from a constant. 
By Proposition 3.2, all left-separable from F^^ time series have the form 




Note that F^' is an L-periodic series with 

E/^'^=0 (22) 

n=0 

if and only if it has the representation as above, see [17, §6.1]. In the case of real- valued time series, 
(2) 

F^ is a sum of harmonics with period L. 

Example 3.2. [Separability from a complex exponent] Let fn^ = A". Then 2^^\F^^) is spanned by 
the single vector (1, A, ... , A^~^)^, and all signal roots of Fj^^ have to be roots of the polynomial 

Wx{z) = X^~\^-^ + ... + Xz + l. 
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Making the change of variables z = x/A, we have to solve VFi(x) = 0, where W^i(x) is defined in (21). 
Therefore, the roots of W\{^z) are simple and have the form 

exp(2-Kik/L) X f 27rik\ 

— J — =^^"H"^J' 

where < k < L. Hence, the time series is of form 

L-l L-l 



k=i k=i ^ ^ ^ 



where € C are not simultaneously zero. This representation is more illustrative if we represent the 
roots fik in the polar form. If A = pexp(27ria;), p > 0, u G [0; 1), then 

Pk = exp (^Tii (^^ + ^ • 

The roots pk a-re shown in Fig. 2 




Figure 2: Roots of the series left-separable from a complex exponent. 

Let us write out the results for the case Im A = 0, when F^'^ is a real exponential time series. If 
A = Re A > (i.e. uj = Q), then fn'^ = /)" is left-separable from all time series of type fn^^p~^fn\ 
where fn^ has period L and zero sum (in the sense of (22)). If A < (w = 0.5) then f^^ = (— p)" is 
also an exponentially modulated cosine with period 2 (an exponentially modulated saw-tooth series), 
and the set of all left-separable series can be described in a similar way. 

Let us consider the important case of a sum of conjugate exponents, which corresponds to real 
cosine time series with periods other than 2, multiplied by a complex constant. This case has some 
peculiarities. 

Example 3.3. [Separability from a sum of conjugate exponents] Let fn'^ = cA" + dA", Im A ^ 0. We 
choose ImA > 0, without loss of generality. Then, by Example 3.2 and Remark 3.3, any signal root p 
of F^^ must satisfy 
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where < k,l < L. Hence, 



A / 27rim\ 



= = exp 
A 



where m 1j. Therefore, A has to be of form 



A = pexp{^^J, p>0, (23) 



for to be separable from . Since ImA > we can choose m such that < m < L. Then the 
signal roots of F^^ are of the form 



-1 /^27ri, , A 
Hk = P exp I ^(2fc - m) I 



where < k < L, k ^ m, and the time series has the representation 



fi'^= E c.p-«exp(g(2fc-m: 



Note that, in contrast to Example 3.2, the constraint (23) on the signal roots of F^^ naturally 
arises. In the real-valued case, if two exponentially modulated cosine series are left-separable, then 
these cosines should have integer periods related to L. Indeed, if fn^ = p" cos((/3 -|- Inun) = cA" -|- dX^ 
where ip £ M., p > and uj G (0,0.5), then lo must satisfy co = m/2L, < m < L, for F^'^ to be 

(2) 

separable from another time series. Thus all real- valued time series Fj^ that are left-separable from 
F^^ have the following representation 



/F= E c,p""cos(^, + 2vri^^n) 



where fco = [y\ , 



ke{l...,L-5}\{ko} 
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0, m is even, 

1, m is odd. 



Cfc, (fk are arbitrary reals and not all Ck are zero. This is in agreement with considerations in [17, 
§6.1], but the present exposition is more illustrative. 

(2) 

Note that we have two slightly different situations for even and odd m. In essence, p^fn has to be 
2L periodic; however, for even m it necessarily has the period L. This effect has been already observed 
(for the cosine time series) in [J 7, §6.1], but in the present paper, again, the nature of this effect is 

(2) 

more evident. Examples of locations of signal roots of Fj^ for these two situations and p = \X\ = 1 
are depicted in Fig. 3. 

Finally, let us show that in the case of multiple roots no time series is separable from a given time 
series. 

Example 3.4. [Separability from an exponentially modulated polynomial] Let fn^ = Pi(n)A"', A 7^ 0, 
and Pi{n) be a polynomial of nonzero degree. Then, by Proposition 2.3, the basis of F^^ includes the 
vectors 



1^A = ^^(A) = (1,A,...,A^ 
4(A) = (0,l,2A,...,(L-l)A^-2^^_ 
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Figure 3: Roots of the series (small dots) separable from a sum of conjugate exponents (large dots), 
p=l. 



Note that in this basis we can replace t\{X) by 



<^ = ^l(A) + = (1, 2A, 3A2, . . . , LX^-^f. 



X 

(2) 

By Proposition 3.2, all signal roots of should be at least common roots of the generating poly- 
nomials W\{z) and W^\z). Making the change of variables z = x/X, as in Example 3.2, we have to 
solve 

Wi{x) = Wl^\x) = 0, 

where Wi{x) is defined in (21) and Wi^\x) = Lx^~^ + . . . + 3x^ + 2x + 1. Wi{x) has the roots on 
the unit circle, whereas W^\x) has decreasing positive coefficients, and therefore, by the Enestrom- 
Kakeya theorem [1], its roots lie strictly inside the unit disc, and hence W\{x) and W^\x) have no 
common roots. Thus, no f.d.d. time series is left-separable from 

3.3 Enumeration of separability cases 

First, let us explicitly enumerate all cases of the separability between nonzero f.d.d. time series. By 
Example 3.4, only time series with the simple signal roots can be left-separable. Consider two time 
series 

i=i 

such that Cfc, dj ^ and ^ Aj, pj ^ pm for k ^ j ^ m. Then, by Remark 3.3, these time series 
are left-separable if and only if each summand c^A^ is separable from each djp^. Summarizing all the 
above and using Example 3.2, we obtain the following proposition. 

Proposition 3.3. Two f.d.d. time series F^^ and F^^ are left-separable if and only if they have the 
form (24), and there exist p > and uj € [0; 1/L) such that 

( ( 1TT-k 

Xk = pexp y2TTi + ^ 
Pj = p~^ exp ^27rz + u 

where < mk,nk < L are distinct numbers. 
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As in Example 3.3, Proposition 3.3 can be specialized to real-valued time series. In this case a 
constraint on L appears again (either a; = or a; = 1/2L). 

Let us examine the separability of backward and forward non-continuable time series. Note that 
hereafter we do not assume L < N/2. 

Proposition 3.4. Let F^^^ and F^^ be two nonzero time series, 1 < L < N, and CL £ 
These time series are left- separable if and only if both are "border" time series, specifically, 

p(i)_/n n f(i) f(i) ^ 

p(2)_.,(2) ,(2) . l^^J 

where 1 < d < L. 



Proof. Let cl S H^^^F^"^). Then the columns X^^''^^ of the trajectory matrix X.^^\F^^) satisfy 

'^)±eL, 1 <i < K. 
(/f ,...,/i!!2^0,...,0). 



x|^'^^_LeL, 1 < i < K. Hence the last component of each vector xj^^'"^^ is zero and finally F^^ 



K 

Let us consider the reversed time series F^^ and F^\ Since is nonzero, we have e^ G 

£!^^\F^^^) and, applying the first part of the proof, we obtain F^^^ = (/^Ij^, . . . , fx^^^ . . . , 0). The 

K 

proof is completed by noting that the sum of the lengths of the non-zero parts at the beginning of 
F^^ and at the end of F^^ cannot exceed L; the details are left to the reader. □ 

From Propositions 2.7 and 3.4 we immediately obtain the following result. 

Corollary 3.1. Let F^^ and be two nonzero time series, L < N/2, and either F^^ is forward 

(2) 

non-continuable or Fj^^' is backward non-continuable. Then the time series are left-separable if and 
only if they have the form (25). 

Remark 3.4. By Proposition 2.8, the complete classification of all the cases of left separability is 
given by Proposition 3.3 and Corollary 3.1. 

Let us finish this section with the consideration of the conventional, two-sided, separability. Two 
time series are called weakly (two-sided) separable if they are both left- and right-separable [17, Ch. 
6]. 

Proposition 3.5. Corollary 3.1 remains valid if we replace the left separability with the two-sided 
separability, and L with mm{L,K). 

Proof. Without loss of generality we can choose L < N/2. Corollary 3.1 implies that the | =^ | part 
takes place. The | <^ | also takes place since all time series of form (25) with L = Lq are of the same 
form for all L > Lq. □ 

Therefore, we should again consider only the cases of separability of f.d.d. time series. Let F^}^ 

and Fj^^ be time series of difference dimension d^^^ and d^'^\ respectively. Then they are separable 
only if their trajectory matrices do not have full column or row rank, and hence, by Proposition 2.1 
the window length L satisfies the inequality max((i^^\ d*-^-') < L < N — max{d^^\d^'^^) + 1. One can 
see that Proposition 3.2 and Remark 3.3 can be extended to handle these window lengths and hence 
can be applied for both L and K. 

Proposition 3.6. Let L* denote the greatest common divisor of L and K . All examples from Sec- 
tion 3.2 and Proposition 3.3 are valid if we replace the left separability with the (two-sided) weak 
separability and L with L* . 
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Proof. It is sufficient to prove tfie proposition only for Example 3.1, since the other assertions are 
based on it. In this case the roots Hk from Example 3.1 have to be common roots of ^1(2) defined in 
(21) and the polynomial z^~^ + . . . + 1 = {z^ — l)/{z — 1). One can immediately observe that fik are 
of form 

f2-Kik\ 

Hk = exp ( -J— 1 , <k < L , 
which completes the proof. □ 



4 Extraneous roots of SSA continuation LRF 

In the present section we study the behaviour of the extraneous roots for the specific LRF which 
is used in SSA forecasting, see [17, §5.2]. For an f.d.d. time series we express this LRF through 
the characteristic polynomial of the time series. Then we show the correspondence between the 
extraneous roots and a special system of orthogonal polynomials. Using this correspondence, first, 
we demonstrate that several main properties of the extraneous roots are easily proved and, second, 
we derive the asymptotic behaviour of the extraneous roots for the f.d.d. time series in the noise-free 
case. 



4.1 SSA LRF and its basic properties 

Let A be a subspace of such that e/, ^ A. Let {Ui, . . . , U^} Q be the orthonormal basis of A 
and Uk= ( ^J: ) where G C^"^ and vr^ G C. Define 7^ = (ao, . . . , ai-iY G C^"^ as 



1 d 

^ = r37;2E^^^^ (26) 

fc=i 

d 

where v"^ = |vrip + ... + Ivr^P = ^ |(C/i,eL)p < 1. The last inequality holds since cl A. The 

4 = 1 

following proposition is a version of [17, Ch. 5, Th. 5.2] for complex-valued time series. 

Proposition 4.1. Let be a time series of difference dimension d, 1 < d < L. Let A = £^^^(F/v). 
Then satisfies an LRF 

L-l 

fn+L = <^kfn+k: 0<n<iV-L-l, (27) 
fc=0 

with the coefficients given by (26). 

The LRF (27) is abbreviated as the SSA LRF. Proposition 4.1 provides the base of the SSA con- 
tinuation algorithms, see [17, Ch. 2]. In these algorithms one finds A that approximates £^^^(F/v) and 
employs the SSA LRF obtained from A, which approximates the SSA LRF obtained from 2^^\Fn). 
A detailed discussion of this approximation can be found in [22, 23]. 

Let R be the conjugate to the orthogonal complement of A (c.f. Definition 2.3). The next propo- 
sition is a version of [J 7, Ch. 5, Prop. 5.5] for the complex-valued case. 

Proposition 4.2. The vector 

A = {-n\ Vf = (-ao, . . . , -aL_i, l)\ (28) 

with TZ given by (26), can be expressed as 

A = cIlKeL, (29) 

where XIr is the orthogonal projector on the space R and c = (1 — u"^)'^ = (11^6^,6^)"^ . 
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By Proposition 4.2, the SSA continuation vector (28) is equivalent to the Min-Norm prediction 
vector [5, 9]. One of its weU known properties, providing the name Min-Norm, is the following; the 
proof can be found, for example, in [4]. 

Proposition 4.3. The vector (28) yields the minimum of 

|aoP + |aiP + ... + |aL_ip (30) 

among the vectors from R. 



4.2 SSA LRF and the characteristic polynomial 

Let us fix a time series Fat of difference dimension d with characteristic polynomial P{z). Let L be 
such that d<L<N — d+1 and A = 2,^^\Fn), i.e. the noise-free case is treated. Then Hfy^ = Hr 
is the projector on the relations space IH, see Definition 2.3. Consider the vector B = Hsj^eL. By 
Proposition 4.2, A = cB. Let A{z), B{z) denote the generating polynomials of A, B, see the notation 
in Section 3.1. By Proposition 2.2 we obtain 

A{z)/c = B{z) = P{z)Hn{z), (31) 

where Hn{z) = ht^z^^. . . + is a polynomial of degree n+1, n'^L — d—1. The n extraneous 

roots of A(z) are exactly the roots of the polynomial Hn{z). Below we study the properties of these 
polynomials. 



Proposition 4.4. The vector 



is given by 



Hr. = {h^-\...M:^y (32) 



H„ = fP*P)-ie 



n+l 



where P = p(^) is defined in (18) and P* denotes the Hermitian conjugate of P. In other words, 
is the unique solution of 

T^nHn = Cn+l, (33) 

where 

rj.^dcfp*p g (^(n+l)x(n+l)_ (^34^ 

Proof By Propositions 4.2 and 2.2, B = P^^^Hn and minimizes the Euclidean norm 1 1 P ^ ^ — ei, 1 1 2 
among all vectors V G C^~'^. Therefore, Hn can be expressed as the least squares solution of P^^^V ~ 

Hn = (P*P)-ip*eL = {P*P)-^en+u (35) 
where the last equality holds since = 1. □ 

Remark 4.1. In fact, the projector Yl^ itself can be explicitly expressed through the characteristic 
polynomial as liy^ = P(P*P)-ip*. 
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By construction, the matrix defined in (34) is a ToepUtz Hermitian matrix 

J-n — 

with 



tk= { 



( d-k 

PjPk+j, <k <d, 

j=0 

0, k > d, 

[tl^, k<0. 



(36) 



Note that the coefficients do not depend on n. Also, any matrix T„ has no more than 2d + 1 
non-zero diagonals 



/to ••• t. 



\ 



td 



to 



\ 



td 



to J 



By the way, tk coincide with the values of the covariance function for the moving average process 
with coefficients pk, see [2, Ch. 5]. If we rewrite the equation (33) as 



TGn = ei, 



(37) 



where G = {hn , • • • , y , then we obtain the Yule- Walker equation for this process. 



4.3 Orthogonal polynomials. Basic properties 

In this subsection we apply the powerful theory of orthogonal polynomials on the unit circle and 

obtain short proofs for the basic properties of extraneous roots. Let us first rewrite the equation (33) 

d 

in terms of orthogonal polynomials. Let t{z) = ^ t^z^, where tk are given by (36). It is easy to see 

k=-d 

that t{z) = P{z)P{l/ z) and the relation 

t{z) = P{z)P{z) = |P(z)p > (38) 

holds for all z € Ti, where = {z € C : |z| = r} denotes the circle of radius r. 

For a non-negative function w{z) € Li(Ti) (i.e. a Lebesgue integrable over the contour Ti function) 
one can define the inner product in the space of complex polynomials 

{p{zlq{z))J^^ I p{z)'^)w{z)d9, (39) 

where z = e^^ . The function w{z) is called the weight. In particular, one can define the inner product 
(•, ■)t for the weight t{z). 

Proposition 4.5. Polynomials Hn{z), n > 0, defined by (31) form an orthogonal system with respect 
to i;-)^ 
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equation (33) as 



Proof. It is evident that tk-i = (l,^ = \z ,z for all k,l G Z. Therefore, we can rewrite the 



' C; I 1. = n. 



Hence {Hn{z), Hm{z))^ = for m 7^ n and 

||F„(z)||?<i^f(F„(z),/7„(z)), = /i(r)/0 

for all n > 0, where /in"^ is given in (32), and the assertion is proved. □ 

Remark 4.2. Let <^„(z), n > 0, be a system of polynomials of degree n, which are orthogonal with 
respect to some weight w{z) (i.e. {(^n{z),(^rn{z))^ = for m 7^ n, ||<I>„(z)||^ 7^ for all n). Then 
are defined uniquely up to constant factors. This follows from the properties of the standard 
orthogonalization process applied to the sequence {l,x,x^, . . .}, see [13, Ch. IX, §6]. 

Thus, Proposition 4.5 is a characterization of Hn{z). 

Remark 4.3. The system from Remark 4.2 is also orthogonal with respect to the weight aw{z) 
for all a > 0. 

Now let us recall the well-known property of orthogonal polynomials, which has a consequence for 
the extraneous roots. We provide the proof just to show its simplicity, it also can be found in [IS, Ch. 
!]■ 

Theorem 4.1. Let ^n{z) he an orthogonal polynomial of degree n with respect to (•, ■)w Then \zq\ < 1 
i/$n(zo) = 0, i.e. the roots of the polynomial are located inside the unit circle. 

Proof. If has a root zq then ^n{z) = Q{z){z — zq), where degQ = n — 1, and hence Q)w = 0. 
Then 

\\Q{z)\\l = {zQ{z),zQ(z))^ = \\zQiz)\\l = 

= \\Q{Z)Z0 + ^niz)\\l = \zom{z)\\l + \\^n{z)\\l, 

where the last equality follows from the orthogonality of polynomials. Rewriting we have (1 — 
|zoP)||Q(-z)||2; = ll^n(^)||w > 0) which completes the proof. □ 

In particular, we obtain the following result for the SSA LRF (Min-Norm prediction), which is 
well known [ i, .5, 15], but the proof is often too complicated. 

Corollary 4.1. All extraneous roots of SSA LRF lie inside the unit circle. 

Next, we provide another clear proof for the well-known result [15] on the correspondence between 
the extraneous roots for the forward and backward SSA LRFs. By the backward SSA LRF we mean 
the SSA LRF for the reversed time series F^; the forward LRF means the standard SSA LRF. It is easy 
to observe (for example, using the basis in Proposition 2.2) that F/v is an f.d.d. time series with the 
characteristic polynomial Pq^P(z), where P{z) denotes the reversed polynomial + . . .+poz'^. 
This polynomial has roots A^^ with multiplicities (c.f. (8)). For convenience, for any polynomial 
B{z) = brZ^ + . . . + biz + 60, br 0, we introduce the notation 

B* {zf^^%{z) = b^z'' + ... + b^iz + Vr. (40) 

Then 

\P{z)\^ = \P*{z)\^ = \P{z)\\ \z\ = l. (41) 
This fact enables us to easily show the following. 
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Proposition 4.6. The extraneous roots of the backward and forward SSA LRFs are conjugate. 

Proof. Let u{z) = \pq\~'^\P{z)\'^ denote the weight for the reversed time series. Then for any Q{z) and 
S{z) we obtain 

{Q{z),S{z))u = \por\Piz),Q{^))t, 

which follows from (39). Therefore, each polynomial from a set of polynomials orthogonal with respect 
to u{z) has the roots which are conjugate to the roots of the corresponding Hn{z), and the assertion 
is proved. □ 

Note that it is common to define the backward SSA LRF with conjugation of the coefficients [5]. 
In this case, the assertion of Proposition 4.6 changes; specifically, the extraneous roots of the backward 
and forward SSA LRFs coincide (as stated in Introduction). 



4.4 Asymptotic properties 

In this section we present a review of recent results on the asymptotic distribution of the roots of 
orthogonal polynomials which can be used to study the extraneous roots for the SSA LRF. We mainly 
follow [20] and [21] and present all the facts in a unified and simplified manner, specializing the results 
for the weight t{z) defined in (38). 

For convenience, in addition to we define 

I])r{af^{z G C : |z - a| < r}, 

B/=??D^(0), 

Jif^iz G C : lz| < r}, 

for r > and a S C. 

Let us consider the characteristic polynomial P{z) defined in (8) and the orthogonal polynomials 
Hn{z) defined in (31). Using a transformation similar to (41) we can "transfer" all the roots inside 
the closed unit disk Bp Define 

c{z)= n (^-A.)'^^- n (42) 

fc:|Afc|<l ;:|A,|>1 

Then 

\P{z)\'' = c\C{z)\\ (43) 

where c is some positive constant. Hence, by Remark 4.3, the weight t{z) = |P(2;)p and the weight 
u{z) = \C{z)\^ generate the same system of orthogonal polynomials Hn, and the roots of C{z) are 

inside the closed unit disk. 

Note that in the representation (42) the pairs of roots of P{z) which are related by = A^~^ are 
glued together. The normalized representation of C(z), defined in (44), takes this into account. We 
set 

C{z) = {z-air^ ...{z-asT^ (44) 

where all a^, 1 < A; < s, are distinct. 

Let us also introduce several definitions. The radius p = maxi<fc<s \ak\ < 1 is called the critical 
radius for the polynomial P{z). The circle Tp is called the critical circle. Moreover, let the roots of 
C{z) be ordered such that the first u roots are on the critical circle and the other roots are inside it, 
i.e. |ai| = ... = \au\ = p and \ak\ < p for k > u. We will call ai, . . . ,au the leading roots of C{z). 
Let also the first i roots ai, . . . ,ae be of greatest multiplicity M among ai, . . . , a^, i.e. = M for 
all I < s < i and mj < M for £ < j < u. 
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Proposition 4.7 ([20, Prop. 1, Th. 3]). Let p < I. 

1. For any e > there exists -/Vi(e) such that all zeros of Hn{z) are inside Op+e for all n> Ni. 

2. For any e > there exists N2{e) such that the closed disk Up.g contains at most i — 1 roots of 
Hn{z) for all n> N2. 

Proposition 4.8 ([21, Cor. 1]). If p = 1 then for any e > there exists iV2(e) such that for all 
n > N2 the closed disk B]^_^ contains at most u — 1 roots of Hn{z). 

These two propositions imply that the majority of roots tend uniformly to the critical circle Tp 
(they are called general roots), and a bounded number of roots stay strictly inside Bp. (called spurious 
roots). The spurious roots chaotically float inside Dp (when changing n), but, roughly speaking, they 
are asymptotically close to zeros of the functions G„, defined by 



Gn{z) 



" a^+'i+i(_l)-.^,(C*)(-'=)(afc) 



^ (z-afc)CK)(afc) 



where /("^^ stands for the m-th derivative of a function / and C*{z) is defined as in (40). See [.3, 20, 21] 
for exact formulations; see also an example with spurious roots at the end of this section. 

Let us look at the behaviour of general extraneous roots. We present only an informal summary 
in order not to overload the exposition with technicalities. Precise and mathematically strict results 
can be found in [20, Th. 4] and [2.1, Th. 5] (for p < 1 and /O = 1, respectively). 

• The asymptotics for the absolute values of the general roots are 
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n \n I 



For a function /, define by Zg{f) = Ua7(a)=o e-vicinity of its zero set. Let us denote 



ULiID'e(afe), p = l. 



Then for large n and small 6 there are no spurious roots in the area 

Ae,5,n = {z:\\z\-p\<6}\ {Bc,e U Z,{Gn)). 

• Let P be a connected sub-area of Ae^s^n and z^\ . . . , Zr"^ denote the roots of H^{z) in V, ordered 
by the magnitude of their arguments. Then for i,j such that 1 < i, {i + j) <r we have 

arg(4"^.) - arg(zf ^) = ^ + O 

These results can also be found in [3] in a weaker form. Let us make some comments. 

• The general roots show an asymptotic angular equidistribution. The order of their convergence 
to the critical circle differs for p = 1 and p <1. In the latter case the convergence rate depends 
on the maximum multiplicity M of the leading roots of C {z) . 
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• For p < 1 only the roots ai,...,ai (i.e. the leading roots with maximum multiplicity M) 
significantly affect the asymptotic behaviour of extraneous roots: they determine the number of 
spurious roots, and their vicinities are excluded from Ae,s,n', however for /> = 1 all leading roots 
of C{z) affect the asymptotic behaviour. 

• In the real- valued case each signal root is usually accompanied with its conjugate (when the 
signal is a sum of modulated cosines). Therefore, there is likely to be more than one leading 
root of C{z), and the spurious roots are likely to appear. 

Example 4.1. [General and spurious roots] Consider a sum of two exponentially modulated (with 
the same exponent) cosine time series. Since each cosine corresponds to two conjugate roots, we have 
£ = u = 4 and M = 1. The roots of the SSA LRF are plotted in Fig. 4 (recall that L = n + d+l). 





Figure 4: Signal (x) and extraneous (o) roots, fn = 0.9" (cos {^n) + cos (27r sin(0.25)n)) 

As one can see in Fig. 4, the spurious roots float inside the critical circle; for some L the spurious 
roots disappear. The coefficient sin(0.25) was chosen to ensure the rational independence of the leading 
roots of C{z). If the majority of leading roots is rationally dependent (i.e. some roots are expressed 
through linear combinations of others with rational coefficients), then the spurious roots float more 
regularly, see [3, 20, 21] for details. 

4.5 Several applications and remarks 

First, let us show a connection between separability and the behaviour of extraneous roots. We 
demonstrate that the approximate [17, Ch. 6] (left) separability of a linear time series from a periodic 
time series can be informally justified by the condition of separability developed in Section 3. Let 
W^^'°\xf:^^Wi{x) and W^^''^\zf:^wi^\z) be the polynomials from Example 3.4. The roots of W^^'^^ 
are always on the unit circle and uniformly distributed with equal angles between adjacent roots on 
it (without number 1). One can show that W^^'^\x) for L = 1,2,... form an orthogonal system 
with respect to the weight P{z) = \z — 1|^, and by results in Section 4.4 the roots of W^^'^\x) are 
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distributed asymptotically in the same way as the roots of W'^^^^\x). In Fig. 5 one can see the roots 
of T^(^'0)(a;) and W^^^^\x) for large L. 




Figure 5: The roots of and W^^'^\ 

We might say that this validates the asymptotic separability of an arbitrary periodic series from 
a linear function. The same can be shown for any polynomial time series. Note that this observation 
does not pretend to be a rigorous assertion. A recent theoretical study of asymptotic and approximate 
separability can be found in [23] . 

Let us now consider the problem of determining the signal roots from the approximate SSA LRF 
(the Min-Norm estimation of the main roots). Theorem 4.1 is the base of the root- Min- Norm [4, 9, 12] 
method for estimation of exponents. Under the assumption that all signal roots have modulus not 
less than 1, one selects the greatest by absolute value d roots from the estimated LRF to be the signal 
roots. 

If all signal roots have the absolute values greater than 1, we conclude from the asymptotic dis- 
tribution of the roots that the root-Min-Norm approach is applicable. In the presence of moderate 
noise, the main and extraneous roots of the estimated LRF are close to those of the noiseless LRF, 
see, for example, [4]. In Fig. 6 a noisy time series with its SSA LRF roots are depicted. If some signal 



L= 50, N= 100 




20 40 60 80 100 -1.0 -0.5 0.0 0.5 1.0 



Figure 6: Left: the time series. Right: the signal roots (x) and the extraneous roots of the SSA LRF 
(o), /„ = 1.05" + 3^1.1" cos(0.5n) e„ are i.i.d. AA(0,502), 

roots are on the unit circle, then this approach behaves worse. 

If all signal roots have modulus less than 1, then the naive root-Min-Norm approach is not appli- 
cable. Even if the signal roots are only on the critical circle, the extraneous roots may have larger 
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modulus, as shown in Fig. 7. In this case one can use the backward SSA LRF to get the main and 
extraneous roots separated, see [4]. 

Assume now that within signal roots there are roots both outside and inside the unit circle. In 
this case one can consider the backward and forward SSA LRF simultaneously and use the relations 
between the extraneous and signal roots of both LRFs. 

Finally, we discuss the notion of multiplicity. Consider the case of a single real exponent time 
series, modulated by a quadratic polynomial. There is one signal root of multiplicity 3. In Fig. 7 
the roots of SSA LRF for the noise-free case and the noise case are depicted. As one can see in 




Figure 7: The signal roots (x) and the extraneous roots of the SSA LRF (o), /„ = + std • En, 

En are i.i.d. J\f (0,1), N = 150. 

Fig. 7, in the presence of noise a multiple signal root splits into three separate roots. Nevertheless, 
the extraneous roots behave as if there was a multiple signal root. Indeed, the extraneous roots are 
close to those in the noise-free case. In addition, no "spurious" roots appear inside the critical circle. 



References 

[1] GoviL, N.K. and Rahman, Q.I., (1968). On the Enestrom-Kakeya theorem. Tohoku Mathemat- 
ical Journal, 20:2, 126-136. 

[2] Anderson, T.W., (1971). The statistical analysis of time series, John Wiley & Sons, New York. 

[3] SzABADOS, J., (1979). On some problems connected with orthogonal polynomials. Acta Mathe- 
matica Academiae Scientarium Hungaricae, 33:1—2, 197-210. 

[4] KUMARESAN, R. and Tufts, D.W., (1982). Estimating the Parameters of Exponentially 
Damped Sinusoids and Pole-Zero Modelling in Noise. IEEE Transactions on Acoustics, Speech, 
and Signal Processing, 30:6, 833-840. 

[5] KuMARESAN, R., (1983). On the Zeros of the Linear Prediction-Error Filter for Deterministic 
Signals. IEEE Transactions on Acoustics, Speech, and Signal Processing, 31:1, 217-220. 

[6] Heinig, G. and ROST, K., (1984). Algebraic methods for Toeplitz-like matrices and operators, 
Akademie Verlag, Berlin. 

[7] Pakula, L., (1987). Asymptotic Zero Distribution of Orthogonal Polynomials in Sinusoidal Fre- 
quency Estimation. IEEE Transactions on Information Theory, 33:4, 569-576 

[8] Mhaskar, H.N. and Saff, E.B., (1990). On the distribution of zeros of polynomials orthogonal 
on the unit circle. Journal of Approximation Theory, 63, 30-38. 



23 



[9] Krim, H., Forster, p. and Proakis, J.G., (1992). Operator approach to performance analysis 
of root-MUSIC and root-min-norm. IEEE Transactions on Signal Processing, 40:7, 1687-1696. 

VAN DER Waerden, B.L., (1994). Algebra I, 9th ed., Spinger Verlag. 

Elsner, J. and TSONIS, A., (1996). Singular Spectrum Analysis. A New Tool in Time Series 
Analysis, Plenum Press, New York. 

StoicA, p. and Moses, R.L., (1997). Introduction to spectral analysis, Prentice Hall. 
Gantmacher, F.R., (1998). The Theory of Matrices, Chelsea, Providence, RI. 
Hall, M. Jr., (1998). Combinatorial theory, Wiley, New York. 

Bezerra, L.H. and Bazan, F.S.V., (1998). Eigenvalue Locations of Generalized Companion 
Predictor Matrices. SIAM Journal on Matrix Analysis and Applications, 19:4, 886-897. 

Nekrutkin, V.V., (1999). Approximation spaces and continuation of time series. In S. M. 
Ermakov and Yu. N. Kashtanov (Eds.), Statistical Models with Applications in Econometrics and 
Neibouring Fields, University of St. Petersburg, St. Petersburg, 3-32 (in russian). 

GOLYANDINA, N., Nekrutkin, V. and Zhigljavsky, a., (2001). Analysis of Time Series 
Structure: SSA and Related Techniques, Chapman & Hall/CRC, Boca Raton. 

Simon, B., (2005). Orthogonal Polynomials on the Unit Circle, Part 1: Classical Theory, AMS 
Colloquium Series, American Mathematical Society, Providence, RI. 

Badeau, R., David, B. and Richard, G., (2006). High-Resolution Spectral Analysis of Mix- 
tures of Complex Exponentials Modulated by Polynomials. IEEE Transactions on Signal Pro- 
cessing, 54:4, 1341-1350. 

Martinez-Finkelshtein, a., McLaughlin, K.T.-R. and Saff, E.B., (2006). Szego Orthog- 
onal Polynomials with Respect to an Analytic Weight: Canonical Representation and Strong 
Asymptotics. Constructive Approximation, 24:3, 319-363. 

Martinez-Finkelshtein, A., McLaughlin, K.T.-R. and Saff, E.B., (2006). Asymptotics of 
orthogonal polynomials with respect to an analytic weight with algebraic singularities on the 
circle. International Mathematics Research Notices, 2006:91426, 1-47. 

GOLYANDINA, N., (2010). On the choice of parameters in Singular Spectrum Analysis and related 
subspace-based methods, http://arxiv.org/abs/1005.4374. 

Nekrutkin, V., (2010). Perturbation expansions of signal subspaces for long signals. 
http://arxiv.org/abs/1001. 1051. 



24 



